On Context-Sensitive Usability Evaluation in Mobile HCI

In usability evaluations, experiments are often conducted in closed laboratory situations to avoid disturbing influences. Due to non-realistic usage assumptions, this approach has important shortcomings when mobile Human Computer Interactions (m-HCIs) have to be evaluated. On the other hand, field studies allow to perform experiments close to real-world conditions, but potentially introduce influences caused by the environment, which have not been fully investigated so far. With this work, we contribute to distinguishing application shortcomings from environmental disturbances which both potentially cause decreased user performance. Our approach is based on monitoring environmental conditions during the usability experiment, such as light, acceleration, sound, temperature, and humidity, and relating them to user actions. Therefore, a mobile context-framework has been developed based on a Wireless Sensor Network (WSN) carried together with a mobile PC. We present results of a small study (seven persons) in the lab which pointed at increased delays and error rates of user interactions under induced environmental disturbances. Hereby, we demonstrate the potential of environmental monitoring for understanding user performance. Additionally, we present novel results of a usability study carried out in the field where we tested 19 persons under varying environmental conditions. The results showed that error rate and delay are influenced by environmental parameters, but in a more complex way than expected a-priori.